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Introduction 

The communities of scientists who study the Earth's climate 
and the atmospheres of the other planets barely overlap, but the 
types of questions they pose and the resulting implications for 
the use and interpretation of remote sensing data sets have much 
in common. Both seek to determine the characteristic behavior of 
three-dimensional fluids that also evolve in time. Climate 
researchers want to know how and why the general patterns that 
define our climate today might be different in the next century. 
Planetary scientists try to understand why circulation patterns 
and clouds on Mars, Venus, or Jupiter are different from those on 
Earth. Both disciplines must aggregate large amounts of data 
covering long time periods and several altitudes to have a 
representative picture of the rapidly changing atmosphere they 
are studying. This emphasis separates climate scientists from 
weather forecasters, who focus at any one time on a limited 
number of images. Likewise, it separates planetary atmosphere 
researchers from planetary geologists, who rely primarily on 
single images (or mosaics of images covering the globe) to study 
two-dimensional planetary surfaces that are mostly static over 
the duration of a spacecraft mission yet reveal dynamic processes 
acting over thousands to millions of years. 

Remote sensing displays are usually two-dimensional 
projections that capture an atmosphere at an instant in time. 

How scientists manipulate and display such data, how they 
interpret what they see, and how they thereby understand the 
physical processes that cause what they see, are the challenges I 
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discuss in this chapter. I begin by discussing differences in 
how novices and experts in the field relate displays of data to 
the real world. This leads to a discussion of the use and abuse 
of image enhancement and color in remote sensing displays. I 
then show some examples of techniques used by scientists in 
climate and planetary research to both convey information and 
design research strategies using remote sensing displays. 

Novices versus Experts 

The problems beginning student;; face in interpreting remote 
sensing displays have much in common with those they encounter 
with any mathematical representatior. of the real world: 

(1) Novices often cannot translate their own observations 
of the real world into more abstract: representations of the 
larger-scale picture. For example, after a lecture on the forces 
that determine the flow around low and high pressure, and how 
that relates to the isobars displayed on a weather map, students 
in an introductory climate course at Columbia University and 
Barnard College went outside to measure pressure and wind 
direction. Afterwards, the students were shown a schematic 
weather map with a high-low pressure pair and corresponding 
isobars, including arrows indicating the approximate flow. Fewer 
than half the students could identify where on that map we were 
minutes earlier when we watched the wind carry a balloon away, 
noted the sky condition, and confirmed that pressure was rising. 
This presages the problems that more advanced students have in 
trying to express physical statements as mathematical equations, 
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or to see the ramifications of an equation or symbolic diagram 
for the behavior of the physical system it describes. 

(2) Novices tend to seek out isolated features of any data 
set they examine, and rarely think in terms of the superposition 
of several phenomena to explain what they see. For example, the 
students in the same class were asked to draw a graph of the 
temperature one would measure on a three-day trip from New York 
to Florida in winter. Many students drew a simple straight line 
with an increasing trend. Some added an oscillatory pattern to 
indicate day-night changes. Only a few recognized that there 
would also be random day-to-day changes as weather systems 
passed. Later in the semester, the students were given a data 
set showing the change in Earth's global surface temperature over 
the past 100 years. Asked to describe the graph's salient 
features, most could only say that temperature had risen over the 
past century and attributed this to rising concentrations of 
greenhouse gases. Few noticed that from 1940-1965 temperatures 
had actually cooled slightly, a possible indicator of ocean 
circulation effects or solar luminosity changes that had been 
discussed in class. Fewer mentioned the random year-to-year 
variability superimposed on the pattern. 

These general difficulties carry over to the realm of remote 
sensing displays. Given an image of a planetary atmosphere or a 
false-color contour map of a climate parameter, the novice, with 
no background in the physics of the system under study, often 
simply does not know what to look for and what questions to ask. 
The situation is something like opening the look "Where's Waldo?" 
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and trying to find Waldo without having seen what he looks like 
first. The difference between novice and expert here seems to be 
that the expert goes in with a mental image of what the remote 
sensing display might look like, based on his/her current 
understanding of the system, and thus can react to the actual 
display with questions about how it departs from expectations. 

As an example, students in the climate system course are 
asked to interpret Earth Radiation Budget Experiment (ERBE) 
false-color maps of the geographic distribution of absorbed 
sunlight (Figure 1). They tend to focus on the decrease of 
sunlight from equator to pole (indicated by the transition from 
red/orange in the tropics to blue in the polar regions), which is 
fundamental but is of little interest to the researcher, who 
takes that well-understood aspect of the map for granted. The 
expert instead focuses on less obvious longitudinal variations in 
absorbed sunlight over a homogeneous surface such as the ocean, 
which are diagnostic of poorly understood variations in clouds 
that are of more interest. In Figure 1, e.g., the subtle 
transition south of the equator from the deeper reds in the mid- 
Pacific and mid-Atlantic to orange just off the west coasts of 
South America (20°S, 70°W) and Africa (15°S, 10°E) reveals the 
presence of climatically important low-level stratus cloud decks 
in the eastern oceans. The expert starts with a mental model 
(the intensity of incident sunlight varies with latitude because 
of Earth's spherical shape and axial tilt) and a resulting 
expectation (uniform solar heating at a given latitude over the 
ocean because the ocean is approximately equally reflective 
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everywhere) , and looks for deviations from the expectation to 
learn something. The novice, without a prior mental model, 
simply "draws" a mental caricature of the display that emphasizes 
the major features and misses the details. 

A related bias of the novice is to pay inordinate attention 
to an obvious or spectacular feature of a satellite image, at the 
expense of the more numerous but more amorphous features that are 
more important indicators of the climate (the same is apparently 
true of novice interpretation of weather maps [7]). Students, 
for example, enjoy studying pictures of extreme weather events 
such as hurricanes. But these storms are rare and affect a 
relatively limited area, so their contribution to the seasonal 
mean temperature and total rainfall of a region is usually 
negligible. In the absence of such storms, most students would 
guess that skies are clear. But on average, the sky is not much 
less cloudy over high pressure centers than over the low pressure 
centers we associate with storms. Mainly the type of cloudiness 
changes, from deep thick rain clouds in storms to innocuous 
cirrus or low stratus or fair-weather cumulus before or after a 
storm. Storms are present a small fraction of the time, so the 
more frequent fair-weather, cloudiness may be just as climatically 
important, but this is rarely anticipated by the novice. 

When we image other planets, the playing field becomes more 
level for the expert-novice comparison. Our understanding of the 
other planets is primitive compared to that of the Earth. Most 
planets have cloud features that do not resemble those of the 
Earth. Figure 2 shows Venus, for example, devoid of the swirl: ng 
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midlatitude storm clouds so well-known to us on Earth. We 
understand this as a result of Venus’ slow rotation period (243 
days) , which eliminates the instability that produces our 
midlatitude storms. So we expect not to see such features on 
Venus. But in the absence of that process, what other processes 
prevail on Venus? Can we see evidence of them in the image? At 
first glance, the eye is drawn to the planet-sized dark feature 
shaped like a sideways letter "Y" that spans the equatorial 
region. Analysis of several years of such images by the author 
has identified this feature as a particular type of wave and 
documented its characteristics [3] . Unfortunately, I concluded 
that this feature is secondary among the processes that maintain 
the surprisingly strong winds that blow on Venus. The most 
important processes occur below the visible clouds, hidden from 
our view. We have been fooled into overinterpreting what we see, 
because it is all we have to go on! In other words, the other 
planets can make novices out of experts because their behavior is 
at least partly unfamiliar to all of us. 

The challenge, then, is to manipulate and display remote 
sensing data in ways that optimize transmission of information, 
whether in a research or an educational setting. In the next 
section I describe some common approaches to this problem. 

Image Enhancement and Color 

The image of Venus just discussed does not represent how 
Venus would look to the human eye. Venus is covered with whitish 
sulfuric acid clouds; it looks like a featureless tennis ball. 
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However, other sulfur compounds in the clouds absorb ultraviolet 
(UV) radiation, and when viewed through a UV filter, places where 
such compounds are lifted up above most of the bright sulfuric 
acid veil show up as slightly dark regions in a gray-scale image 
of reflected UV light, which is the Figure 2 image. In the "raw" 
version of this image, even the dark areas are so faint as to be 
barely detectable by the human eye, both because the contrasts 
are inherently small and because sunlight falling on a sphere 
produces larger variations in brightness over the disk, i.e., the 
image is bright where the Sun is directly overhead, and darkens 
toward dawn or dusk. The latter effect impedes scientific 
analysis of the individual features that reveal Venus' weather. 
Thus, planetary scientists routinely first use a computer 
algorithm to subtract out global brightness variations due to the 
different angles at which sunlight falls on different locations. 
This in turn requires us to know enough about the clouds of Venus 
to anticipate how well sunlight is reflected as a function of the 
angle at which it enters and the angle at which we view it. 

The next challenge is to enhance the weather features that 
are really present. This depends on what one wishes to see. In 
Figure 2 we can see both dark regions that span the planet, such 
as the "Y, " and local cloud blobs and streaks reminiscent of 
things we can see in our own sky. To emphasize one or the other, 
we filter the image, i.e., we mathematically separate the 
brightness variations into small and large spatial scales, 
mathematically boost the strength of the scales we wish to see, 
and then reconstruct the image. This is the visual equivalent of 
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turning up the bass vs. the treble on one’s sound system. In 
Figure 2 a high-pass filtered version of the image (which 
emphasizes the small features) was added back to the original 
image (in which the "Y" dominates), to produce a product in which 
all spatial scales are visible. This makes the image more 
pleasing for public relations purposes, but it also enhances the 
scientist's insight - the more irregular nature of small features 
within the dark arms of the "Y" compared to the more linear 
nature of the small features in the bright regions to either side 
contains clues about the processes that form the features and the 
stability of the underlying atmosphere. This is an example of 
constructive image manipulation. 

Not all such efforts are as successful. Black-and-white 
imagery is frowned upon by the public relations offices at NASA 
centers. On a planet such as Venus that doesn't cooperate by 
providing much color, one must resort to false colors. A 
misguided attempt of this sort can be seen on the March 29, 1974 
cover of Science magazine, which showed a UV image of Venus 
obtained by the Mariner 10 spacecraft. This image was false- 
colored blue and white (with the dark regions in Figure 2 
appearing as blue) to resemble "blue planet" pictures of Earth. 
The result is visually striking but scientifically misleading (as 
are many of the planetary images seen by the public [11]): On 

Earth, clouds (which are white) form in updrafts, while the blue 
ocean below is visible mostly in places where air sinks and 
clouds evaporate. On Venus, dark places instead are where the UV 
absorber is lifted above the bright sulfuric acid, while the 
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bright places are likely to be regions of descending motion. 

Thus, a terrestrial bias led to a display that obscured the 
proper scientific interpretation of the image. 

Sometimes a planet does contain intrinsic color, though, and 
false color can be used to create a dramatic and less misleading 
image. The Voyager 1 image of Jupiter’s Great Red Spot and 
surrounding regions in Figure 3 is a good example. This image 
has a certain basis in truth, because it is a superposition of 
three monochromatic images sensitive to different colors. But 
not all colors were included in the Voyager camera filters, so 
the image is biased. The Jupiter that one views through a 
telescope is really much more yellow-green and lower in contrast 
[11]; the Great Red Spot is really a pale pink. Of more concern 
is how the color scheme influences one’s interpretation. Dark 
regions such as that between the Great Red Spot and regions of 
swirling white clouds to the west look to the human eye like 
clearings between the clouds in which we see to great depth, and 
are thus regions of sinking air. Two decades later, this 
interpretation is still open to question. Recent Galileo images 
of lightning on Jupiter suggest that the canonical view of where 
air is moving up and down on Jupiter may have to be revised [5]. 

Aside from images, false color is used to display contour 
maps of remote sensing retrievals of weather and climate 
parameters. It is often overlooked that remote sensing retrieval 
techniques are not perfect - photons emitted or reflected by a 
planet and received by a detector in space are converted by a 
computer algorithm to the numerical value of some physical 
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parameter. The algorithm used may be simply empirical, based on 
observations at one location that are not valid globally. It may 
assume that other physical parameters that also affect the 
radiation observed are constant, when in reality they are 
varying. Or the algorithm may be a physical model of the 
atmosphere that makes simplifying assumptions. But the result, a 
color contour map with a title indicating the "parameter" 
displayed (e.g., precipitation) immediately attains an air of 
legitimacy and certainty among scientists who know but sometimes 
forget that it is not a direct measurement of raindrops collected 
in a bucket. The intervals chosen for changes in color sometimes 
have little relationship to the inherent uncertainty in the 
algorithm that produced the data set, and users often do not ask 
how large the errors are. This is a different take on the novice 
vs. expert issue - professional users of data may be naive about 
how the data being displayed were produced, and hence erroneous 
scientific conclusions may sometimes be reached. 

The preferred choice of colors depends on the quantity being 
displayed. Sometimes the choice seems obvious based on common 
associations - e.g., red for warm and blue for cold on a 
temperature map, or brown for drier and green for wetter on a map 
of drought severity. When a particular color scheme is not 
obvious, an effective choice is to color extreme high values red, 
which draws the eye's attention. Maps of differences (e.g., 
between a satellite data set and a climate model prediction, or 
between El Nifio years and the mean climate [8] ) are commonly used 
to highlight changes in the real world or errors in a model. A 
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useful color choice in such maps is white for the interval 
spanning zero, in which changes/differences are too small to be 
of concern because they are physically unimportant or within the 
error bar of the data. Figure 4 shows such an example. The left 
side shows a climate model prediction of the coverage of cirrus 
clouds, with the red areas drawing the eye to the cloudiest 
places on Earth. The right side shows differences between the 
model's prediction and the cirrus cloud cover inferred from 
satellite remote sensing of visible and infrared radiation. It 
is immediately obvious where the model seriously over- and 
underestimates cirrus. The spatial pattern of the differences 
suggests possible causes of the problem to the model developer. 

Of more importance than color choice is the parameter 
interval over which the color changes. Consider precipitation, 
which varies by an order of magnitude from the tropics to the 
deserts and polar regions. The novice might choose to display 
this field using, e.g., 6 colors with equal spacing at 
precipitation intervals of 2 mm/day, producing a map that 
resembles a more exaggerated version of the left side of Figure 
4. For the novice, such a display conveys information, namely 
that rainfall is highest near the equator and on the western 
sides of ocean basins. To the expert, though, this is well-known 
and therefore of no research interest. A nonlinear scale that 
resolves 0.5 mm/day differences in dry regions while sacrificing 
detail at the high end reveals subtle but interesting variability 
at middle and high latitudes relevant to drought occurrence and 
climate changes in ocean currents. 


This example points out two other differences between the 
novice and the expert. The expert is aware of what is already 
known, and selects display characteristics to bring out the 
unknown, while the novice starts with no such knowledge base and 
thus makes default choices for a display. Also, novices tend not 
to appreciate that percent differences are more important than 
absolute differences, and thus that a 1 mm/day anomaly in a 
region whose mean rainfall rate is 2 mm/day is more significant 
than a 2 mm/day anomaly in a region whose mean rainfall rate is 
10 mm/day. Similar thinking pervades the general public - 100 
point daily swings in the Dow-Jones Industrial Average were news 
when the Average was near 1,000, but it has taken the media many 
years to realize that a 100-point swing in today's 10,000-point 
Dow is cause for neither alarm nor excitement. 

With these general concepts as background, we next discuss 
some of the specific technigues that scientists use to manipulate 
and display remote sensing data to understand Earth's climate. 

Earth Remote Sensing for Climate Research 
Displays of the Atmosphere's Variation with Time 

Satellite imagery is a static display of a dynamic field, 
and it is the dynamics that best reveals the underlying physical 
processes. Remote sensing scientists use a variety of techniques 
to gather and represent dynamic information. A climate expert 
can deduce a great deal just from the morphology of clouds in a 
single image. For example, a well-defined comma-shaped cloud 
pattern is good Evidence of the mature stage of a strong 
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midlatitude storm. But patterns in single images are sometimes 
ambiguous - an amorphous cloud pattern may either be the 
beginning stage of such a storm or a minor disturbance that never 
organizes into something stronger. Only animation of a series of 
images can distinguish the two possibilities. 

The shape of a midlatitude storm cloud pattern is 
immediately interpreted dynamically by the expert, who recognizes 
the tail of the comma as the cold front, knows that southwesterly 
flow of warm humid air and northerly flow of cold dry air (if the 
storm is in the Northern Hemisphere) usually lie to the east and 
west, respectively, and sees the cloudiest regions as locations 
of upward motion. The novice may recognize the pattern shape but 
make no dynamical associations. Understanding can be enhanced 
for the novice by superimposing other fields on the image, e.g., 
vectors indicating wind direction and strength and color contours 
displaying temperature. However, unless a prior mental model of 
the dynamics exists, the observer has difficulty integrating the 
individual parts into a comprehensive picture [7], 

Sometimes there is no alternative but to display the time 
dimension itself. Increasingly, animation is used to track 
movements of features and to reveal interactions between 
different geographic regions. Only in a movie can one appreciate 
the "rivers" of upper troposphere moisture that originate in the 
tropics and flow all the way to midlatitudes. Figure 5, for 
example, shows a GOES water vapor image. It is obvious to the 
novice and expert alike that high humidity along the east coast 
of the United States on this day is spatially linked to 
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convection (the bright specks of humidity seen throughout the 
tropics) occurring near 10°N, 120°W, and that a similar 
relationship exists between convection west of Hawaii and the 
Pacific Northwest United States. The expert has a mental model 
that transforms this static display into a dynamic 
interpretation: Updrafts in tropical thunderstorms loft moisture 

to high altitude, and the prevailing winds near the tropopause . in 
the tropics move this humid air poleward, ultimately affecting 
the sky condition and weather in the United States. The novice, 
with no mental model, cannot tell whether the tropics is 
affecting the midlatitudes or vice-versa, or whether the pattern 
just forms as a single entity at all locations at the same time. 
By animating the images, though, tie novice can watch variations 
with time in the tropical convection and see how their effects 
move along the "river" to higher latitudes. The cause-effect 
link is established via the animation. 

Another important aspect of time variation is characterizing 
propagating wave features. A common way to do this is to 
construct a Hovmoller diagram: A latitudinally thin (5-10°) 

strip covering a broad range of longitudes is extracted from a 
satellite image. A similar strip from the next day's image is 
placed directly above, and the process is repeated for perhaps a 
month or more. The result is a composite image in which many 
two-dimensional arrays of longitude vs. latitude have been 
converted into a single display of longitude vs. time. In a 
Hovm&ller diagram, a propagating wave shows up as lines of clouds 
tilted with respect to the horizontal, with the sense and angle 
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of tilt indicating the direction and phase speed of propagation. 
Persistent stationary features are solid vertical lines, while 
non-propagating oscillations are the image equivalent of a dashed 
vertical line, with the length of the dashes indicating the 
period of oscillation. False color Hovmoller displays of 
tropical Pacific monthly sea surface temperature over several 
years have become a common means of portraying the onset and 
decay of El Nifio [8] . 

Figure 6 shows Hovmbller diagrams of GOES visible, water 
vapor, and "window" (outside the wavelengths of strong water 
vapor absorption) infrared imagery for a region in Oklahoma over 
the month of June 1999. Storm cloud patterns (the bright 
features in the middle and right images) tfilt from lower left to 
upper right, indicating that the storms propagate from west to 

east with time. The degree of tilt tells us how fast the storms 

/ 

move. For example, the storm that originated near 98 °W on June 
28 had moved to 95.5°W (a distance of about 225 km) by the end of 
the day, giving a propagation speed of about 2.6 meters/sec. 
Another advantage of the Hovmdller diagram is the ease with which 
the observer can mentally assimilate missing data. Blue lines in 
the image are times at which the satellite did not acquire or 
transmit useful data, yet even the novice should be able to guess 
what the missing data would have looked like by mentally 
connecting the pattern below the blue line to the pattern above. 

As mentioned earlier, real atmospheres are a superposition 
of processes occurring on different temporal and spatial scales. 
Separation of the different contributions ~o what we see can be 
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difficult for novices and experts alike, especially when a subtle 
long-term climate trend must be detected in a parameter that 
fluctuates more strongly on shorter time scales. Mathematical 
techniques such as empirical orthogonal function (EOF) analysis 
separate space-time data sets into a few characteristic modes of 
variation [6]. The EOF itself displays the spatial pattern; it 
is accompanied by a function of time, the principal component 
(PC) . The decomposition is mathematical and is not guaranteed to 
provide physically meaningful results, but the combined display 
of the EOF and the PC can often be interpreted as a physical 
process by the expert. For example, an analysis of monthly sea 
surface temperature (SST) anomalies has a first EOF showing 
anomalies of opposite sign north and south of the equator, with 
an accompanying PC that is sinusoidal with a period of one year. 
To novice and expert alike, this is obviously capturing the 
seasonal cycle. The second EOF may show large anomalies in the 
tropical east Pacific and a PC that oscillates somewhat 
irregularly with a time scale of a few years; this is El Nifto. 

One research value of such a decomposition is that if the 
dominant features of the EOF and PC suggest a particular physical 
process to the person viewing the display, unanticipated spatial 
relationships can be discovered. For example. Figure 7 shows the 
first PC and EOF of the 20th Century record of precipitation for 
rain gauge stations over all land areas of the globe, after 
subtraction of the seasonal cycle [1] . The first PC time series 
is seen to be well-correlated with observed SST anomalies over 
the east equatorial Pacific, which are known to be caused 
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primarily by El Nifio. The corresponding EOF shows some well- 
known geographic effects of El Nifio: Negative precipitation 

anomalies (drought) over eastern Australia, the Amazon Basin, and 
equatorial Africa, and wet anomalies over the western half of the 
United States. But the EOF also suggests more subtle 
"teleconnections" to places thousands of miles away that were not 
previously known to be affected by El Nifio, e.g., wetter than 
normal conditions in Europe and western Asia. The second PC (not 
shown) is an upward trend over the century that might be an 
effect of anthropogenic global warming but which is undetectable 
in a simple animation of the data due to the large El Nifio 
signal. The corresponding EOF indicates that midlatitudes have 
become wetter and the tropics a bit drier, over the past century. 

The Use of Displays in Computer Algorithm Development 

As noted above, short-term climate changes such as El Nino 
produce large weather perturbations in certain parts of the world 
that are clearly noticeable to people (e.g., the flooding in 
usually dry California in 1997-98). Long-term anthropogenic 
global warming due to building concentrations of greenhouse gases 
is more gradual and smaller in magnitude. Thus it is detectable 
only as a subtle change in the frequency of occurrence of unusual 
weather, e.g., more or fewer droughts or strong storms over 
decades. These changes may be barely noticeable, but 
collectively they have the potential to impact society via the 
availability of water, the growth of crops, the spread of 
diseases, erosion of shorelines, and summer electricity demand. 
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Remote sensing scientists use satellite data to understand how 
the climate responds to changes in temperature, but such changes 
are much smaller than day-to-day fluctuations in weather. The 
human eye-brain combination cannot sense such changes reliably, 
since (a) it notices the obvious, spectacular aspects of an image 
at the expense of the more mundane but numerous features that 
weigh more heavily in determining the climate change, (b) many 
images spanning seasons or even years must be analyzed to get a 
statistically significant result, (c) visual displays of 8-bit 
data, which resolve 256 brightness levels, cannot easily 
communicate net shifts of only a few brightness levels that 
represent the collective sum of slight brightenings in some 
pixels and slight darkenings in others. 

Climate researchers therefore resort to the computer to 
tally statistics for them in an unbiased, representative fashion. 
But the scientist needs to program the computer to look for the 
right thing. A current project involving New York City high 
school students and teachers working with NASA scientists seeks, 
for example, to determine whether clouds in midlatitude storms 
are systematically different in either visible brightness, cloud 
top height, or coverage under warmer versus colder conditions, 
and in years when the temperature difference between the tropics 
and poles is larger versus smaller. To do this we require a 
program that recognizes and tracks storms as they grow and decay; 
this is done objectively using ancillary data on surface pressure 
patterns to locate storm centers. How big an area of the 
satellite image should we include in our definition of a storm, 
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though? In other words, where does a storm begin and end? For 
some storms, with vigorous cold fronts and clear skies behind, 
the rear boundary of the storm is obvious. For others in which 
low-level cloud "debris” remains after frontal passage and 
gradually dissipates, the choice is more problematic. Pattern 
recognition techniques are not yet sophisticated enough to 
encompass all possible storn cloud morphologies, especially the 
less organized ones. 

Thus, the student observers determine a plausible storm 
definition by visually examining a number of images and we 
incorporate their subjective impressions into the computer 
algorithm that compiles the statistics. We then must address 
Mark Twain's "lies, damn lies, and statistics" warning about 
being skeptical of what we cannot see. We run the computer 
algorithm for a month of images and use displays to see whether 
it "detects" anything our eyes tell us isn’t really a storm or 
misses things we would identify as storms. As a final check, we 
can test the sensitivity of any conclusions to the assumptions we 
have made by re-running the program with an altered storm 
definition. In this way, the human and the computer work hand- 
in-hand to do what neither can do alone. 

Displaying Vertical Structure in the Atmosphere 

Another challenge for Earth remote sensing is the vertical 
dimension. Images project a 3-D world into 2 dimensions, and the 
scientist must know what altitude is being observed, because 
understanding sometimes depends on knowing how things going on at 
one level in the atmosphere are influenced by things happening at 
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a lower level or at the surface. Atmospheres usually work from 
the bottom up - the warm dense lower atmosphere is the "dog" 
wagging the "tail" that is the cold, tenuous upper atmosphere. 

For example, an increase in cirrus clouds near the tropopause 
observed in isolation cannot be interpreted, but that observation 
combined with the knowledge that temperature or humidity near the 
Earth's surface had increased at abcut the same time might enable 
the scientist to explain the change in clouds, since the expert 
knows that warm moist air near the ground destabilizes the 
atmospheric column, and that thunderstorms that originate near 
the ground and transport moisture to high altitudes are the 
atmosphere's response to this. 

The problem, though, is that satellites view a planet top 
down, and we can't always see through the complete atmosphere. 
Climate scientists deal with this difficulty in a number of ways: 
(1) By simultaneously observing at two wavelengths, one a 
"window" of weak absorption in which the atmosphere is 
transparent and we see to the surface, the other a wavelength of 
strong absorption by atmospheric gases in which we "see" only 
part of the way down, we can sense both top and bottom. (2) By 
looking up from the ground with surface-based remote sensing 
instruments, we see lower levels better locally but not globally. 

(3) Clouds are at least partly opaque to the short and moderate 
wavelengths of most planetary radiation, so "active” remote 
sensors such as radars, which emit their own long-wavelength 
radiation that penetrates through clouds, now fly on spacecraft. 

(4) The atmosphere changes faster than the Earth's surface 
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beneath it, so we can observe the surface by waiting for the 
clouds to clear. 

Figure 8 shows an example of the latter technique, a single 
frame of an animation of 3 months of operational weather 
satellite data [10]. The analysis uses visible and window- 
infrared images to distinguish clouds from the underlying 
surface. The visible channel analysis makes use of our knowledge 
that clouds are brighter than most surfaces, and the infrared 
analysis uses the knowledge that cloud tops are usually colder 
than the surface because temperature decreases upward in Earth’s 
troposphere. Figure 8 uses four color bars, one each for clouds, 
ice/snow-cover, continents, and open oceans, with different hues 
for each type and different levels of saturation distinguishing 
higher vs. lower temperature. The simultaneous polar projections 
allow us to compare clouds in the two hemispheres. When 
animated, even beginning students are able to detect temporal and 
spatial relationships between surface and atmosphere, e.g., the 
increase in high thick tropical clouds in the mid-afternoon when 
the temperature of the land surface is warmest, and the 
association of persistent high cloud in the equatorial west 
Pacific with the warm sea surface temperatures there. 

This works as long as one can distinguish cloud from 
surface. In the polar regions this is difficult, because snow 
and ice are just as bright as clouds in the visible and the 
atmosphere can be warmer than the surface, which can confuse the 
interpretation of infrared images. The apparent absence of 
clouds near the North and South Poles in Figure 8 is partly a 
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failure to detect them. Such regional errors in remote sensing 
data sets wreak havoc among climate scientists who are remote 
sensing novices. At one extreme, the naive user accepts Figure 8 
as fact and worries needlessly that his climate model produces 
"too much" polar cloudiness. At the other extreme the skeptic 
who knows that the error bars in the polar regions are large, 
dismisses good data in other parts of the world and refuses to 
use the data set at all. In my experience, both types exist in 
the supposedly objective, meticulous scientific community. 

A recent trend is to use multi-spectral microwave emission 
to separate cloud from snow/ice effects in the polar regions, 
since higher microwave frequencies are more sensitive to clouds 
and lower frequencies to sea ice [4]. Unfortunately, the 
separation is not complete, and ambiguity remains. Thus, at any 
instant a bright region in an image may be a cloud, an ice- 
covered ocean location, or ice hidden beneath a cloud. But when 
animated, both novice and expert are able to use the different 
morphology of cloud systems and sea ice and the different time 
scales of variation (days for clouds, weeks for sea ice) to 
separate the two. The observer can see a spiral cloud shape over 
the dark ocean disappear as it passes over ice and then reappear 
over ocean downstream and link the two as the same entity. The 
observer can separate the primarily north-south seasonal growth 
and decay of sea ice from the primarily west-east movement of 
cloud systems. Whether such interpretation capabilities can be 
translated to the computer to enable the processing of years of 
such data remains to be seen. 
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Challenging as it is to understand Earth remote sensing 
data, we at least have the advantage of everyday experience and 
routine surface weather observations to guide our development of 
effective display strategies. When we remotely sense another 
planet, though, the same rules may not apply, and our approach 
therefore is somewhat different. 

Remote Sensing of Planetary Atmospheres 

In principle, remote sensing of other planets is no 
different from remote sensing of Planet Earth, but in practice 
this is not the case. The volume of data on all the other 
plartets combined is a small fraction of that for the Earth, 
although the advent of long-term planetary orbiters has finally 
begun to create a data volume issue in planetary research. In 
any event, our conceptual understanding of other planetary 
atmospheres still greatly lags that of our own atmosphere. 
Consequently, even the expert is something of a novice, 
attempting to make sense of cloud patterns that bear little 
resemblance to those we see on Earth (Figures 2, 3) . The expert 
draws upon terrestrial experience to interpret these cloud 
features. For example, the Voyager imaging team routinely 
referred to isolated small white clouds (such as that below and 
to the left of the Great Red Spot of Jupiter in Figure 3) as 
"convective clouds", i.e., scientific jargon for thunderstorms, 
because of their resemblance to such features in Earth imagery. 
But this association implies the presence of a specific type of 
instability, and in truth there was no independent corroborating 
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evidence to assure us that convection was indeed the process 
producing the clouds. Theoretically, we expect convection to 
exist sporadically on Jupiter, but to this day we cannot 
demonstrate why those clouds appear different from the 
surrounding larger-scale clouds tinged with pink, brown, or 
blue/gray. Depending on the final outcome of this story, the 
expert's experience may turn out to have been a boon or an 
obstacle to true understanding. 

The other major difference between Earth and other planetary 
atmospheres is that most of the latter are completely cloud 
covered (Venus, Jupiter, Saturn, Titan, Uranus, and Neptune) and 
we therefore either cannot see the surface at all, or no solid 
surface exists. This has two implications. First, since 
atmospheres work from the bottom up, what we see with remote 
sensing on other planets is only the tail of the dog, and it is 
hard to know what the rest of the dog looks like. Second, since 
atmospheres are dynamic entities, we have no fixed reference for 
mapping, and thus there is no such thing as a "map of the world" 
for planetary atmospheres. Figure 9 shows an attempt to create a 
rectangular world cloud "map" of Venus by digitally combining 
Pioneer Venus images spaced to correspond to the mean rotation 
rate of the "Y" feature [2], The result is partly a success, 
partly a failure (note the discontinuities at image boundaries), 
because from one day to the next individual clouds grow and 
decay, and because the rotation speeds of the atmosphere and its 
waves are different at different latitudes. This is less of a 
problem on Jupiter, where the rotation rate of the planet itself 
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far exceeds any fluctuations due to winds. 

As on Earth, we use multi-spectral imaging to sense 
different altitudes of other planetary atmospheres, but with 
greater ambiguity. UV filters show sunlight scattered from small 
haze particles at high altitudes in the stratosphere, while 
near-infrared filters probe beneath the high hazes to sense 
sunlight reflected from clouds deep in the troposphere. Longer 
wavelength thermal infrared radiation senses the planet's own 
emission, which increases with temperature and therefore depth 
below the highest cloud tops. Maps of such thermal features 
side-by-side with images of reflected sunlight tell the expert 
whether a specific cloud feature is higher or lower in the 
atmosphere, or perhaps not a cloud at all. We can observe a 
planet with a filter that admits radiation at a wavelength at 
which a specific gas is known to absorb (e.g., methane band 
imaging of the Jovian planets) . The higher the cloud top, the 
less overlying gas there is to absorb, and therefore the brighter 
the feature in the image. Finally, we can image the nightside of 
a planet in visible filters, looking for lightning flashes that 
mark the location of thunderstorms at great depths that might 
otherwise be hidden from our view by the overlying cloud layers. 
Figure 10 shows such an example from the Galileo mission to 
Jupiter. Although this image is a view of the nightside, one can 
see both the lightning flashes from deep levels and the clouds at 
upper levels, because the latter were weakly illuminated by 
moonlight from Jupiter's closest satellite Io. 

In all of these Cc-.ses, interpretation is almost impossible 
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for the novice. It depends on having not just a mental model of 
the weather of the atmosphere, but also a mental model of how 
specific types of electromagnetic radiation interact with a 
specific type of atmosphere. The expert must know the source of 
the radiation (reflected sunlight versus the planet’s emitted 
heat) , the composition of the atmosphere and whether the gases 
present absorb or not at the wavelength being observed, and 
whether the atmosphere contains small haze particles (which 
reflect short wavelengths well but long wavelengths poorly) as 
opposed to large cloud droplets (which reflect all wavelengths 
well but may be hidden below th«,. reflective hazes in a short- 
wavelength image) . If there arc multiple haze/cloud layers of 
different particle size and composition (as is the case in most 
planetary atmospheres) and a heterogeneous planetary surface, the 
interpretation can be ambiguous even for the expert. Is a near- 
infrared feature in a Hubble Space Telescope image of Titan 
evidence of surface topography or ice, a tropospheric methane 
cloud, or a local thinning of the overlying stratospheric haze? 
Only by comparing images at different wavelengths and tracking 
motions over time can such ambiguities possibly be resolved. 

Since we don't routinely launch weather balloons into other 
planets' atmospheres, most of our knowledge of these planets' 
circulations comes from tracking the motions of cloud features. 
The novice can do a fairly good job of tracking cloud motions by 
observing the same cloud shape in two images acquired several 
hours apart but displayed simultaneously. However, once the 
zeroth-order information has been gathered and higher accuracy is 
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needed to detect subtle variations in the flow, the limitations 
of the novice emerge. For example, planetary-scale cloud 
features (such as the Venus "Y") move at slightly different 
speeds than the small cloud features in Figure 2 because the 
large features are propagating waves, and the up-down motions 
that produce wave crests and troughs look to the eye like 
horizontal motions of the air but are not (imagine, e.g., 
spreading ripples from a rock thrown onto a pond) . It is easy to 
train the novice to focus only on small features more indicative 
of the true wind speed, and to avoid alternating dark-bright 
series of linear features that are obviously small-scale waves. 

It is much harder, though, to train the human eye to detect 
both well-defined cloud "objects" and disorganized fields of 
brightness variation. Thus, even the expert detects only some of 
the possible cloud features that might provide information, 
giving a sparse sample of the wind field. Even worse, since the 
human is drawn to certain types of features and develops an 
expectation of how far and in what direction features should 
move, the resulting wind field may be biased. Finally, in recent 
multi-year planetary orbital missions, thousands of images are 
being obtained, and the time it would take a human to process all 
images from a single mission becomes prohibitive. 

We therefore use automated digital tracking algorithms, 
based on cross-correlations of image brightness fields for 
limited areas of a pair of images, to try and objectively map 
wind speeds in planetary atmospheres [9]. This maximizes the 
information from each image and allows L-rge volumes of data to 
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be processed, giving the statistics needed to answer some current 
research questions. As with the Earth storm-tracking program, 
though, the human and the computer must work together to optimize 
the algorithm. For example, we use visual impressions of cloud 
feature sizes that provide the most reliable wind estimates to 
specify the size of the array of pixels that the algorithm tries 
to cross-correlate. We define a minimum acceptable correlation 
coefficient by visually inspecting the kinds of feature motions 
the algorithm derives under different conditions (ideally we 
would like perfect correlations, but real clouds change shape 
with time) . Most importantly, after the fact we can look at a 
frequency histogram of wind speeds derived by the algorithm. 

This will appear as a well-behaved Gaussian distribution of 
speeds, plus secondary distributions of outliers that are clearly 
not part of the main population of vectors. By visually 
displaying the cloud features that the computer tracked to obtain 
each outlier, we can throw out the spurious vectors (which may be 
due to periodic features that fool the algorithm, large-scale 
brightness gradients not removed in our image enhancement phase, 
etc.) and retain the "real" outliers for in-depth analysis. This 
iterative procedure allows us to refine the algorithm so that 
fewer outliers appear in future versions. 

Unlike terrestrial images, where the expert can visualize 
the dynamics from the static display of a familiar feature, no 
such baseline of knowledge exists for less familiar planetary 
cloud features. Thus, movies are becoming an important tool for 
the planetary fluid dynamicist. Movies of rotating vortices on 


30 


Jupiter and Neptune allowed Voyager scientists to watch 
individual vortices merge or pass by without interacting, and to 
detect individual vortices wiggling as they rotated. This in 
turn provided information about the otherwise unobserved vertical 
structure of these atmospheres. 

Thus far, I have discussed remote sensing display as an 
after-tha-fact exercise: A spacecraft instrument observes a 

planet, the data are processed and displayed, and the scientist 
manipulates and observes the display to understand what the 
planet was doing at the time of the observation. But (albeit in 
a limited way) we can also use displays to look into the future. 
This becomes important when scientists plan specific observations 
they are going to take during a planetary mission. We next 
discuss the use of displays in planning planetary observations. 

The Future: Remote Sensing Displays as Planning Tools 

Unlike Earth-orbiting satellites, which cover the globe and 
have regular, repeating orbits, planetary missions are either 
one-time flybys of one or two satellites or orbiter missions in 
which each orbit has a different shape and orientation to meet 
multiple scientific objectives (e.g., nearly circular vs. highly 
elliptic, closest approach to the planet on the dayside versus 
the nightside, etc.) Instructions to point the spacecraft in a 
specific direction, image through a specific filter, set an 
exposure time, and shutter the camera at a certain moment must be 
relayed to the spacecraft far in advance of the time the image is 
taken (because of human limitations in planning and execution and 
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the finite travel time of radio waves to a distant planet) . The 
Voyager imaging team would not have acquired the image of the 
Great Red Spot in Figure 3 if they had not had some idea in 
advance when and where the Spot was going to come into view. 

Such observation planning can of course be done by examining 
graphs of a spacecraft's latitude, longitude, distance from the 
planet, and time of day as a function of time during the mission. 
What we'd really like, though, is to be able to visualize what 
the planet is going to look like from the vantage point of the 
spacecraft at any future moment, so we know when the most useful 
imaging opportunities arise and what type of observation is 
warranted. Projects such as the Cassini-Huygens mission to the 
Saturn system, already launched for arrival at Saturn in 2004, 
are using computer software that allows that to be done. 

Figure 11 is a schematic example of what Saturn will look 
like to the Cassini Orbiter about a day before it goes into 
orbit. We can visualize what latitude and longitude are in 
direct view, where the day-night boundary is, how good a view of 
Saturn's rings we have at that time, and whether any of Saturn's 
moons are visible. The superimposed square representing the 
camera's field of view tells us whether a single image frame will 
cover the planet or whether we need a mosaic of many images to 
see everything. Once Cassini gets close enough to do an initial 
imaging survey of Saturn and its moons, the schematic diagram can 
be replaced by a projection from a data base of previous images, 
allowing us to use remote sensing displays to predict what Saturn 
and its moons will look like to Cassini at any time. No one is 
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an expert on the future - since atmospheres evolve in time, these 
simulated images will only be educated guesses. But they will 
allow us to optimize the science return by planning the right 
type of image at the right time. 

Conclusion 

Consider three people looking at a yellow layer cake with 
chocolate icing: One has never seen a cake before, another has 

eaten cakes but never made one or seen a recipe, and the third 
has baking experience. Only the third would know most of the 
ingredients used to make the cake. Furthermore, there are limits 
to experiental learning: placed in a fully-stocked kitchen, 

neither of the first two people would be likely to stumble upon 
the recipe in any reasonable amount of time. 

The second person starts out with some advantages over the 
first: Previous experience that although only chocolate icing is 

visible, there is probably cake and a layer of icing or fruit 
inside, and that usually it is either yellow or chocolate cake. 
The first person can advance to the expertise level of the second 
simply by cutting into the cake, looking at the inside, and 
tasting it. Both are then in a position to think about what 
might make the cake yellow, perhaps leading to an inference about 
the use of eggs, and so on, although they will never reach the 
expert level on their own. 

The novice in remote sensing display and analysis is in much 
the same situation. What lessons can be learned to make remote 
sensing display a more useful learning tool? 
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1. ) A prior mental model must exist. It may not be wise to 
have students use displays until they have some basic background 
in the subject. There is also no substitute for curiosity, a 
trait that seems to diminish from childhood to adulthood. 

Novices must be trained first and foremost to ask questions and 
encouraged to "play" with data displays. 

2. ) Flexibility in display is crucial. Most atmospheric . 
remote sensing data are two-dimensional (latitude vs. longtiude, 
or horizontal-vertical cross-sections) and time-varying. It 
should be possible to look at the data from all angles (latitude 
vs. longitude at a given time, latitude vs. time at a specific 
longitude, animations that blend both spatial dimensions with 
time) via menu choices. It should be possible to select subsets 
of the data (e.g., geographic regions) to view in more detail. 
Color bar design should permit selection of a large number of 
colors from a palette, flexible choices of dynamic range, and 
variable data range definitions for different colors. Each of 
these features can be found in existing commercially available 
graphics packages; rarely are all found in the same package. 

3. ) The display software should permit a variety of 
mathematical manipulations of the data; Anomalies with respect 
to a user-defined baseline, frequency histograms, lag 
correlations, spectral analysis, filtering with user-defined 
bandpass; EOF and/or related orthogonal function analysis, and 
mathematical function capability, i.e., the ability to calculate 
and display a function of two or more existing display data sets. 

4. ) Novices must be trained not only how to use all the 
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bells and whistles, but also why one might want to use each one. 
For example, a novice who notices the big midlatitude comma cloud 
patterns but tends to miss the small cloud blobs indicating 
thunderstorms must be trained not only how to use the filtering 
function, but why one might want to notice both the big things 
and the small things in an image. Analogies to other fields 
might help, e.g., the importance of both be.ss and treble to the 
overall impact of music. 

5. ) In meteorology and climate, the whole is greater than 
the sum of the parts. Remote sensing displays should permit the 
user to superimpose multiple fields on the same display. 

Satellite image gray-scale displays with superimposed color maps 
highlighting precipitation, contour maps of pressure and 
temperature, and wind vectors can be a useful way to allow the 
student to think about physical relationships, especially if the 
display can be animated. Simultaneous display of fields at 
different altitudes, such as that in Figure 8, can be used to 
help students see how one part of the atmosphere does or does not 
communicate with another. 

6. ) Novices must be trained to describe what they see in 
more detail than that to which they are accustomed. An exercise 
replicating the experience of someone describing a criminal to a 
police sketch artist might be useful. Two novices, one viewing a 
display and the other not having access to it, work together. 

The first describes the display to the second, whose job it is to 
draw the display just from the description offered by the first. 

The points above highlight the need especially for training 
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documentation written from the standpoint of the novice user, 
rather than from the standpoint of the expert. A common flaw of 
software manuals is that they are so intent on demonstrating all 
the capabilities of the software, they overwhelm the novice who 
wishes to get started doing a few basic things and has no idea 
why the advanced capabilities even exist. Meeting the users 

F 

where they are is and will continue to be the most effective 
strategy for widening the novice-to-expert bottleneck. 
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Figure Captions 

Figure 1. False color contour map of the 5-year mean values of 
solar radiation flux absorbed by the Earth in January (in 
units of watts per square meter) , derived from satellite 
measurements made by the Earth Radiation Budget Experiment 
(ERBE) . 

Figure 2 . An image of the planet Venus taken through an 

ultraviolet filter by the Pioneer Venus Orbiter Cloud 
Photopolarimeter Experiment. The bright regions are clouds 
composed of a concentrated solution of sulfuric acid in 
water. The dark regions are locations in which other sulfur 
compounds have been lifted into view. 

Figure 3. A false-color composite image of the clouds of Jupiter 
taken by the Voyager 1 imaging system. The Great Red Spot 
is visible in the upper right. 

Figure 4. Left: Color contour map of monthly mean cirrus cloud 

coverage (%) simulated by the NASA Goddard Institute for 
Space Studies global climate model for (upper) January and 
(lower) July. Right: Differences in cirrus cloud coverage 

between the simulations at the left and data acquired from 
visible and infrared satellite imagery by the International 
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Satellite Cloud Climatology Project (ISCCP) . The color 
contour resolution of 10% is comparable to the accuracy of 
the ISCCP satellite retrieval of cloud cover. 

Figure 5. Northern Hemisphere GOES water vapor channel gray- 
scale image for April 4, 2000. Brighter pixels indicate 
regions of enhanced upper troposphere humidity and high- 
level cloudiness. 

Figure 6. Hovmoller diagrams constructed from visible (left), 
water vapor charnel (center), and window infrared (right) 
GOES-8 hourly iitagery of northern Oklahoma for June 1999. 

The ordinate indicates time (in days of the month) , 
increasing from bottom to top. The abscissa is west 
longtiude in degrees. Bright areas in the visible images 
represent reflective clouds at any altitude. Bright areas 
in the window infrared indicate mid- and high-level 
cloudiness usually associated with synoptic storms. Missing 
and bad data are indicated by the blue and orange coloring, 
respectively. 

Figure 7. Top: The first PC of the record of observed monthly 

precipitation variations over land for the 20th Century 
(solid line), and the time series of SST anomalies in the 
equatorial east Pacific, an indicator of El Nifio (dashed 
line) . Bottom: The corresponding first EOF of the 20th 

Century land precipitation record, with blue indicating 
wetter than normal conditions at times when the PC 
coefficient is positive, and orange/red indicating drier 
conditions at these times [11 . 
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Figure 8. Northern and Southern Hemisphere polar projection 

false color temperature maps for Oct. 1, 1983, derived by 
the International Satellite Cloud Climatology Project from 
operational satellite infrared imagery. Separate color bars 
differentiate temperatures at cloud tops, ice/snow-covered 
surface regions, oceans, and continents. Temperatures are 
expressed in units of Kelvin, which are identical to degrees 
Celsius but shifted upward in value by 273.15. For example, 
the freezing point of water is 32°F = 0°C = 273.15 K. 

Figure 9. A rectangular projection world "map" of UV cloud 

features on Venus constructed by compositing the central 
sectors of 12 consecutive Pioneer Venus images acquired over 
a 4-day period, the nominal rotation period of planetary- 
scale cloud features. Discontinuties visible at image 
boundaries are caused by evolution of cloud features over 
time intervals of a few hours and by departures of the 
atmosphere's rotation period from 4 days at some latitudes. 
Solid black areas represent missing data. 

Figure 10. Galileo Orbiter visible images of the nightside of 
Jupiter. Bright specks indicate lightning flashes from 
water clouds located at depth below the visible cloud deck, 
while the reddish background shows clouds at higher 
altitudes illuminated by moonlight reflected by Jupiter's 
innermost satellite Io. 

Figure 11. A schematic view of the appearance of Saturn as it 
will be seen by the Cassini Orbiter spacecraft the day 
before it goes into Saturn orbit on July 1, 2004, computed 
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using the CASPER (Cassini Sequence Planner) algorithm 
developed by the project for planning mission observations. 
The superimposed squares are the fields of view of the 
Cassini Imaging Subsystem Wide and Narrow Angle Cameras. 
Dotted areas on the left represent the nightside of the 
planet. "SSP" indicates the point on Saturn directly 
beneath the Sun, and "S/C" the point directly beneath the 
spacecraft. According to CASPER, Saturn's moon Enceladus 
will also be in view just to the lower left of Saturn at 
that time. 
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Fig. 9. Temporal coefficients and spatial patterns of the first leading EOF of annual precipitation 
anomalies (normalized by their standard deviations prior to decomposition). Also shown (dashed 
line) is the bimonthly SST anomalies over the equatorial Pacific fWright 1984 ) . Areas without data 
are in white. The annual PC coefficients (top panel, solid line) were plotted at the SON of each year 
(e.g., 1982 PC coefficient is plotted at x axis = 1982.75 instead of 1982.50) since the largest 
ENSO-induced precipitation anomalies are generally in SON fDai 1996 ). 
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